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(57) ABSTRACT 

A system for distributing load between multiple servers 
where more than one server in a sever cluster receives a 
request for connection from a client and each server makes 
a determination of whether or not to respond to the request. 
Software modules running on the servers monitor and com- 
municate relative abilities of each server to respond to client 
requests. Each server responding to a percentage of client 
requests corresponding to its relative ability to respond. 
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AND M^OD rOH U,AD ^T^^^^rh^n^.'ZS^ 

BALANCING currently available load balancers is that when they 

RELATED APPUCAnON malfunction, their entire server duster becomes inoperaUve. 

• • »^fTTQ nrr^v^ 5 U.S. Pat. No. 6,006,259 reUtes to an I otemet Protocol (IP) 

This appUcation is a network clustering system wherein it attempts to address the 

sional patent application Serial No. 60/169,196 tiled Uec. o, .b-^.^gjjjiojjed drawbacks by distributing the load bal- 

1999, which is incorporated by reference m its entirety, ^^g^^^™^ lo all of the servers in the server 

RACKGROUND OF THE INVENTION cluster. However, it does not address the problem of bal- 

bAUKUKUui 10 ancing the load between non-uniformly loaded servers m the 

The invention relates to the field of data packet manage- ^^^^ ^^^^^ ^ system , considers the available 

ment. More ^ecifically, the invention relates to the regu- ^^^^ /^j. current load) of the server cluster as a group, 

lating data packets lo and from a cluster or group ot data available capacity of the individual servers in the 

servers. server cluster. For example, if one server is operating at 90% 

The evolution over the past 20 years of digital commu- ^.^pacity and another server is operating at 60% capacity, it 

nications technology has resulted in the current distibuted ^ desirable that the load balancing system directs more 

cUent-server data networks, the most well known of which ^^^^^ jQa^jed server. The present mvenUon 

is the Internet. In these distributed client-server networks, advantageously balances load between the servers in the 

multiple clients are able to access and share data stored on ^^^^ ^^^^^ directing traffic based on the available 

servers located at various points or nodes through a given capacity of the individual servers in the server cluster, 
network. In the case of the Internet, a client computer is able 

to access data stored on a server located at any pomt on the OBJECTS AND SUMMARY OF TOE 

planet, as long as the server is also connected to the Internet. INVENTION 

With the rapid proliferation in use of distributed data ^^^^^ ^ object of the present to overcome the 

networks such as the Internet, more and ^o^l^^^^^^^^^^ 25 ^/^^"^^ ab^ve-described systems by providing 

around the world are attemptmg to comiect to and extract technique for balancing or distributing load 

data stored on a finite number of servers^ Those ^tablishing ^^^^"^^^^^^^ cluster. The present invention 

andmaintainingtheserverscontainingthe^reddaus^^^^^ JlTet^rk^^^^^ 

^ZrZ ^nrTcUons a eiven server can support is a functioD a pluraUty of servers, with aU servers haymg the same 

of T^Te^ XuSSrage and commimications Jwork address, and each server having a load balanang 
^;^hS In stations where ttie number of clients 35 module to generate a connection value for each connecUon 

a tfitrto acSSaU sSed on a server exceeds the request A particular server in the server cluster accepUan^ 

^S^pL'y some clients either wiU not be able to processes the network connection request based on the 

^nn«t ^ ^n« dropped by the server. In other cases connection value. That is each server is a^sociated^th a 

wherrl ^r^ is overwhdmed by cUent requests for data. non-overlapping range of connecUon values and accepte 

::rrs°Sh^.t~^^^ L^t^:ierclustertotl«rebydynamic.nyba.ancetheload 

ro^^re^rSte^^::^^^^^^ « •'rLrrinectedtothenetworklnparanel.ch 

rntnSnction^^ the multiple mirrored servers, a load «hat each server receives every^nn^n re^^^^^^ 

balancer is typically used. When a cUent attempts to connect a synchronizmg segment or packet ^Yl^ for transmission 

?o Xc^SLm a server cluster, the client's request control protocol/internet protoc^ (TC^^^^^ -b- 

k first received by the load balancer which determmes stantially at the same time. The SYN packet is me nisi 

whS oTrrserviL is best suited to handle the cUent's 50 segment or packet sent by ttie TCP protocol and is used to 

™t Vario^ load balancing solutions are commercially synchronize the two ends of a comiection m preparaUon for 

aSle and rach^s different techniques and criteria for o'pening a connection. The load balancmg modules on the 

dltermSinfto which server to direct'a cUent's request. servers communicate with each other to determine each 

Ho™fhe common characteristic for each of the cur- server's relative ability to accept a new connection (i.e.. 

renUy available sohitions is that they aU attempt to pick a S5 available capacity). 

server which is most capable of responding to the cUent's Various other objects of the present mvenUon will become 

request readily apparent from the ensuing detailed descripUon ot the 

An inherent drawback with the load balancing solutions drawings. 

» B«.EPDESCR,PnONOP™D.AW«CS 

balancer is the first point of contact for each dient attempt- foUowing detailed description, given by way of 

ing to access data on the server cluster, and theretore, me intended to limit the present invention 
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FIG 2a is a diagram sho"^g the iocorporalion of an pseudo-random number generation function. It is appreci- 

embSit of the Jresen, invention in the Iniemet; a.ed that the load ^alanang module U^ m^^^^^^ 

, . • *• ««Po««tkor conventional random generator to calculate the pseudo- 

FIG 26isadxagramshowmgthemcor^^^^^^^ raoZ nlber. Also, it is appreciated that every server in 

embodiment of the present mvention m the In emet, ^ ^^^^^^ ^^^^^ connection value for each 

FIG. 3 is a diagram showing the sub-range of each server packet, 

in the server cluster of FIG. The relative availability of a server is a function of its 

FIG. 4fl is a block diagram illustrating the flow of a SYN ^^^^^ capacity and current load. It is appreciated that 

Packet within a server according to the present mvention; different versions of the servers may have different process- 

FIG. 46 is a flow diagram illustrating the operation of the jq ing power or capacity. Preferably, an agent program or 

load balancing module of FIG. 4a; application 14 resident on each server collects the availabil- 

FIG. 5 is a diagram showing a client computer connected ity information. The agent 14 can collect the availability 

to the server cluster of HG. 2a; information by monitoring one or more of server's internal 

FIG. 6 is a block diagram illustrating a server cluster conditions which affect the server's ^^nUty to est^^^^ 

according to the present invention within a multi-router 15 connections with the cUent computers 60 THe momtored 

accoramg lo inc pi conditions may include, but are not limited to, the server's 

topology, processing (or CPU) capacity, CPU load, number of con- 

FIG. 7 is a plot of a pseudo-random number generation P;;^^^^;^ J,^ ^e^'g performed, and the number 

^c^ion. existing connections. The agent 14 can also monitor other 

DETAILED DESCRIPTION OF THE ^0 conditions such as the server's internal temperature and 

INVENTION power consumption using instruments connected or resident 

in the server. It is appreciated that electronic temperature 

The present invention is readily implemented using pres- gauges and other instruments capable of transmitting rcad- 

enlly available communication apparatuses and electronic directly to an application resident on a general purpose 

components. The invention finds ready application in virm- computer, i.e., server, are well known to one skilled in the 
ally all data communications system, including but not 

Umited to the Internet, private or public commumcation embodiment of the present invention as shown in 

networks, wireless networks, satelUte networks, cable net- ^ records the availability information of 

works or other online global broadcast networks. .^^ asso^;iat6d server in a pre-defined memory location 16. 

Turning now to FIG. 1, there is ilhistrated an example of 30 ^^^^^ ^4 ^an periodicaUy transmit the availability 

a TCP/IP network incorporating Uic load balancing systena information of its associated server either to other servers in 

of the present invention. It is appreciated that the load cluster (FIG. 2a) or to a coordinating device 20 (FIG. 

balancing system of the present invention is not limited to ^ shown in FIG. 2a, each agent 14 broadcasts its 

the TCP/IP network, it finds ready application in virtually all respective availability information to all of the servers in the 

data networks employing any conventional data protocol, 35 cluster. Alternatively, as shown in FIG. 2b, each agent 14 

such as the system network architecture (SNA), CCCIT transmits its respective availability information only to the 

X.25, etc. A cUent computer 60 is connected to a router 30 coordinating device 20. It is appreciated that the coordinat- 

ihat is connected to servers 10(a)-(c). The client computer . ^^^^ ^0 may be a server or a general purpose computer. 

60 accesses data stored on the servers 10(«H^^^ Tuming now to an example illustrated in FIG. 3 and the 

router 30. Each server 10 has a umque real IP address and 40 ^. ,^ ^^^^ ^^^^^^ ^ 

shares a common virtual IP address with all of je-^^^^ ^^.^^ent on each server 10 generates a range of connection 
the server cluster, e.g., 2.2.2.2. The client computer 60 ^^^^^^ ^ ^ ^^^^^ ^^^^^^^ ^^N packet at step 401. In 
connects to one of the server in the server cluster by ^ packet has a connection value or 

transmitting a <=o^^^^i«°^q^^t\ T^^^^^^^^ c number that is be^een 0 and 32,000, and each connection 

request is in the fomi SYN packeUddressed to the se^^^^^^ 45 ^ ^^^^ ^1^3^,,. 

IP address After receiving the SYN packet from tiie client ^^"^'^ ^^^i' ^ ^ . 

compu^^TeoVth r^er 30 multicasts the SYN packet to all Although, the SYN packet transmuted to each ^rver m 

AfTi,, iMnVJr\ the cluster, only one server, namely the associated load 

Gt the servers it^^Mf? balancing module 12, accepts and establishes a connecUon 

Turmng rK)w to HGS. 2a ^t^^'^f^f ™ P" J^' ,„ ^ the cKent computer 60 based on the connection value of 

received by the server is evaluated by a load balancing SO ^ket The load balancing module 12 on each 

module 12. preferably a software module, ^sident m each ^;^^;^„P1,^3 ^^e connection vie of a given SYN 

server 10. THe evaluation proc^nvolves calculatmg a ^^^r ^°tnparcs connection 

pseudo-random number for each SW p^^^^^ ^V°step^ If the connSuon value of the SYN packet 

ing the relative availabihty of «f "l^^^^ . ^^J"^' „ is within the server's assigned sub-range, the associated load 

balancing module U either pa^ the SYN packet onto the 55 ^ ~ '"^^^^ sYN packet to the TCP 

server's TCP stack or discards the request based on its l^^'^ ^"^^^^^^ j^e connection request from the 

evaluation of the SYN packet. The load balancing module SYNpacket at step 

12 calculates a 4(B. However, if the connection value of the SYN packet is 

value using a portion of the SYN packet as an input. sub-range, the associated 

Preferably, flie p^udo-random number gen«^^^^^^ 60 ^„^~jng module 12 wll then simpl? discard the SYN 

is characterized by an even probability distnbuUon over a " aoa 

fixed range of values and a zero probability outside the pa=«t at step iw. 

range. FIG. 7 shows an example of a sequence of pseudo- For example, m FIG. 3 the server 10(«) accepts SW 

random numbers generated by a suitable pseudo-random packet having a connection value betwe*". i^^sj^^P^J 
number generation function, THe pseudo-random number 65 sub-range of 0 to 10,000, and server 10(b) accepts SW 
generated for each SYN packet may have substantially equal packet having a comie^on value between its ass^ned 
probabilityofbeinganyvaluewithinarangedefinedbythe sub-tange 10.001 to 20,000. If the SYN packet has a 
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connection value of 16,000, then the SYN packet will be real IP addresses to the server each NIC being uniquely 
accepted only by the server 10(/>), and discarded by servers associated with a single real IP address. It is appreciated that 
1 Q/ \ an J each server 10 in the server cluster receives not only packets 

Preferably, the assigned sub-rangc is not static, but addressed to its reallP addresses, but also packets addressed 
dynamicaUy adjusts as a function of each server's availabil- 5 to its virtual IP address. hrn.Hn..t 
itV with respect to the overall capacity of the server cluster. On initiation, the routers 30(a) and 30(f>) broadcast 
Each server's agent 14 continuously or periodically trans- address resolution protocol (ARP) packets to map (or 
mits the server's availabUity information either to other convert) virtual IP addresses, such as 2222 to real P 
^r^^ers in the cluster or to a coordinating device 20, for addresses. ARP packets are generaUy used to dynamicaUy 
example once every two seconds. As this information is 10 T^'^l^^^ ^^^^^^^^^^^ 

received, the sub-range for each server is continuously physical hardware) IP address For example server 10(c) 
recalculated such that The size of each server's sub-range is can answer the ARP with a multicast Ethernet address, such 
SSortion with that server's availability. For example, a fOVT^K"'^?^'^"^ '^''/h WP ^dS^^^ 
seiVer having30%of the total availabQity of a server cluster 10(aH0 subscribe. Thereafter, the rea ,^dre^^c^^^^^ 
would be assigned a sub-range whose size would be about ^5 ^I^^ding to the virtual IP address is stored m the router s 
of the totd range. Given that the comiection values for ARP cache, and aU mcoming packed destined for the virttial 
SYN packets are evenly distributed across the entire range, I? address 2^^ 2 are multicast to the corrcspondmg real IP 
Lt would result in about 30% of the connection values addresses (or Ethernet adckesses). This effectively sends he 
falling within that server's sub-range, and about 30% of all incoming packets to aU of the servers in the server cluster, 
connection being accepted by that server. Other server's in 20 While the present invention has been particularly 
the cluster arc assigned and continually reassigned sub- described with respect to the illustrated embodunent, it wUl 
ranges similarly. If a given server becomes inoperative, the be appreciated that various alterations, modifications and 
sub-ranges of other servers in the cluster wiU expand to adaptations may be made on the present disclosure, and are 
cover the area no longer covered by the down server. Also, intended to be within the scope of the present mvenUon. It 
during the transition period wherein the servers are assigned 2s is intended thai the appended claims be interpreted as 
new sub-range of connection values, a given connection including the embodiment discussed above, those various 
value may be associated with more than one server. In such altematives, which have been descnbed, and all equivalents 
scenario, the connection request may be accepted by two thereto. 

servers, but only one connection will be generally estab- What is claimed: , , ^. , 

lished since most conventional network protocols have 30 1. A system for distributing load vwthm a network, com- 
mechanism to resolve such conflict. For example, under prising: 

TCP/IP protocol, two servers transmit their own SYN at least one cluster having a plurality of servers connected 

acknowledgement (ACK) packets to the client computer 60 to each other, each server in said cluster being addres- 

assodated with the SYN packet. The client computer 60 sable by a common network address; and 

accepts only one SYN ACK packet and rejects the other 35 a load balandng module in each server for calculating a 
SYN ACK packet, thereby establishing a single connection connection value for each connection request addressed 

between the server cluster and the client computer 60, to said network address and for accepting a connection 

In one embodiment of the present invention, each server request as a function of available capacity of said 

receives the availability information from each other server respective server with respect to overall available 

and determines its own sub-range. Based on availability 40 capacity of said cluster and the connection value asso- 
information from its own agent 14 and other agents 14 dated with said connection request, 

assodated with other servers in the cluster, each load bal- 2. The system of claim 1, wherein said connection value 

andng module 12 calculates a new sub-range of connection being within a predefined range of values, 
values that it will accept. Alternatively, the coordinatmg 3. The system of claim 2, wherein said module in said 
device 20 determines and transmits a new sub-range for each 45 each server transmits tiie available capacity of said each 
server based on all of the availability information from each server to other servers in said cluster and determines the 
server 10. The coordinating device 20 or each server in the overall available capadty firom the available capacity of said 
cluster continually adjusts the sub-range for each server to plurality of servers in said cluster. 

reflect the servers' availabilities to accept new connections. 4. The system of claim 3. wherein said module in said 
In accordance with another embodiment of the present 50 each server determines a sub-range within said predefined 
invention as shown in FIG. 6, the servers are arranged rangeof values as a function of the available capacity of said 
within a multi-router topology. That is, the network routes each server with respect to the overall available capacity, 
the data traffic to the servers in the server cluster via multiple each sub-range not overlapping v«th another sub-range, 
routers 30. The multiple-router topology of FIG. 6 increases 5. The system of daim 4, wherein said connection value 
the reUabflity of the network in that the servers can receive 55 being a pseudo-random number; and wherem said module 
traffic from multiple routers. For example, tiie server 10(c) accepts only connection request havmg a connecUon value 
in no 6 is connected to routers 30(a) and 30(b\ thereby within the sub-range of said associated server, 
providing overiapping server coverage. The router 30(fl) 6. The system of daim 2, fiirther comprismg a coordi- 
transmits packets, such as SYN packet, addressed to virtual nating device; wherein said module m said each server 
IP address 2 2 2 2 to the servers 10 via network 12 and the 60 transmits the available capaaty of said each server to said 
router 30(b) transmits padcets addressed to virtual IP address coordinating device which determmes the overall available 
2.2.2.2 to the servers 10 via network 13. Due to the capacity from the avaUable capadty of said plurahty of 
overlapping router coverage, servers may have two different servers in said cluster. 

real IP addresses, e.g., server 10(c) has real IP addresses of 7. The system of daim 6, wherein said coordinating 
123 3 and 13 3.1 respectivdy for routers 30(fl) and 30(f>). 65 device determines for said each server a sub-range within 
It is weU known in the art that multiple network interface said predefined range of values as a fiincUon of the available 
cards (NIC) can be installed in the server to assign multiple capacity of said each server with respect to the overall 
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available capacity, each sub-range not overlapping with 
another sub-range. 

8. The system of claim 7, wherein said connection value 
being a pseudo-random number; and wherein said module 
accepts only connection request having a connection value 
within the sub-range of said associated server. 

9. The system of claim 1, wherein said network is either 
a TCP/IP network or a system network architecture (SNA) 
network. 

10. The system of claim 1, wherein said available capacity 
of said each server is function of at least one of the 
following: server*s processing or CPU capacity, CPU load, 
number of concurrent processes or tasks being performed, 
and number of existing connections. 

11. A system for distributing load within a network, 
comprising: 

at least one chister having a plurality of servers connected 
to each other, each server in said cluster being addres- 
sable by a common network address; 

a plurality of routers for routing connection requests to 
said phirality of servers, each server in said cluster 
connected to said at least one router; and 

a load balancing module in each server for calculating a 
connection value for each connection request addressed 
to said network address and for accepting a connection 
request from a router as a function of available capacity 
of said respective server with respect to overall avail- 
able capacity of servers connected to said router and the 
connection value associated with said connection 
request. 

12. The system of claim 11, wherein said connection value 
being within a predefined range of values. 

13. The system of claim 12, wherein said module in each 
server connected to said router transmits the available capac- 
ity of said each server to other servers connected said router 
and determines the overall available capacity from the 
available capacity of the servers connected to said router. 

14. The system of claim 13, wherein said module in said 
each server connected to said router determines a sub-range 
within said predefined range of values as a function of the 
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available capacity of said each server with respect to the 
overall available capacity, each subrange not overlapping 
with another sub-range. 

15. The system of claim 14, wherein said connection 
5 value being a pseudo-random number; and wherein said 

module accepts only connection request having a connection 
value within the sub-range of said associated server. 

16. A method for distributing load within a network 
comprising at least one cluster having a plurality of servers 

10 connected to each other, each server in said cluster being 
addressable by a common network address, comprising the 
steps of: 

calculating a connection value for each connection 
request addressed to said network address by said each 
15 server, and 

accepting a connection request by a server as a function 
of the connection value associated with said connection 
request and available capacity of said server with 
respect to overall available capacity of said cluster. 
20 17. The method of claim 16, wherein said connection 
value being within a predefined range of values. 

18. The method of claim 17, further comprising the steps 
of: 

transmitting the available capacity of said server to other 

servers in said cluster; and 
determining by each server the overall available capacity 

firom the available capacity of said plurality of servers 

in said cluster. 

19. The method of claim 18, further comprising the step 
of determining a sub-range within said predefined range of 
values for said each server as a function of the available 
capacity of said each server with respect to the overall 
available capacity, each sub-range not overlapping with 
another sub-range. 

20. The method of claim 19, wherein said connection 
value being a pseudo-random number; and wherein said 
module accepts only connection request having a connection 
value within the sub-range of said associated server. 

***** 
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